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^—1 ■ Abstract 

O; 

■ We deal with decoding of Tanner codes using message-passing iterative decoding and 

Q ' linear programming (LP) decoding in memoryless binary-input output-symmetric (MBIOS) 

2 ' channels. We present a new combinatorial characterization for local-optimality of a code- 

C"^^ I word in irregular Tanner codes with respect to any MBIOS channel. This characterization 

I is a generalization of [Arora, Daskalakis, Steurer; 2009] and [Vontobel; 2010] and is based 

L_( i on a conical combination of subtrees in the computation trees. The main novelty is that 

. . the subtrees may have any finite height h (even greater than the girth of the Tanner graph). 

O . In addition, the degrees of local-code nodes are not restricted to two. We prove that local- 
optimality in this new characterization implies Maximum-Likelihood (ML) optimality and 
CN ■ 

^ ■ LP-optimality. Given a codeword and the channel output, we also show how to efficiently 

L ■ recognize if the codeword is locally optimal. 

^ ' We present a novel message-passing iterative decoding algorithm, called normalized 

CN ■ 

• I weighted min-sum (NWMS). NWMS algorithm is a BP-type algorithm that applies to any 

O \ Tanner code with single parity-check local codes (e.g., LDPC codes). We prove that if 

,__( \ a locally optimal codeword for depth h exists, then the NWMS algorithm finds it in h 

^ \ iterations. Hence, the NWMS algorithm has an ML-certificate for any bounded number 

KA . of iterations. Furthermore, since the depth h is not bounded, the guarantee for successful 

S . decoding by NWMS holds even if the number of iterations h exceeds the girth of the Tanner 

graph. 

Finally, we apply the new local-optimality characterization to regular Tanner codes, and 

prove lower bounds on the noise thresholds of LP-decoding in MBIOS channels. When the 

noise is below these lower bounds, the probability that LP-decoding fails decays doubly 

exponentially in the girth of the Tanner graph. 
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1 Introduction 

Modern coding theory deals with finding good error correcting codes that have efficient en- 
coders and decoders ( URUOSID . Message-passing iterative decoding algorithms based on belief- 
propagation (see e.g., [IGal63[|BGT93llMac99[ILMSS01llRU01H ) and linear-programming (LP) 
decoding llFel03[FWK05ll are examples of suboptimal decoders (i.e., may fail to correct errors 
that are corrected by a maximum-likelihood (ML) decoder). 

Many works deal with low-density parity-check (LDPC) codes and generalizations of LDPC 
codes. LDPC codes were first defined by Gallager [|Gal63ll who suggested several message- 
passing iterative decoding algorithms (e.g., "sum-product"). Tanner UTanSlH introduced graph 
representations of linear codes based on bipartite graphs over variable nodes and constraint 
nodes, and viewed iterative decoding as message-passing algorithms over the edges of the 
Tanner graph. In the standard setting, constraint nodes compute the parity function. In the gen- 
eralized setting, constraint nodes use a local error-correcting code. One may view a constraint 
node with a linear local-code as a coalescing of multiple single parity-check nodes. Therefore, 
a code may have a sparser and smaller representation when represented as a Tanner code in 
the generalized setting. Sipser and Spielman IISS96II studied Tanner codes based on expander 
graphs and analyzed a simple bit-flipping decoding algorithm. 

Wiberg et al. [|WLK95[ IWib96ll developed the use of graphical models for systematically 
describing instances of known decoding algorithms. In particular, the "sum-product" algorithm 
and the "min-sum" algorithm are presented as generic iterative message-passing decoding al- 
gorithms that apply to any graph realization of a Tanner code. Wiberg et al. proved that the 
min-sum algorithm can be viewed as a dynamic programming algorithm that computes the 
ML-codeword if the Tanner graph is a tree. For LDPC codes, Wiberg llWib96ll characterized 
a necessary condition for decoding failures of the min-sum algorithm by "negative" cost trees, 
called minimal deviations. 

Linear programming (LP) decoding was introduced by Feldman, Wainwright and Karger 
HFelOBl IFWKOSH for binary linear codes. LP-decoding is based on solving a fractional relax- 
ation of an integer program that models the problem of ML-decoding. LP decoding has been ap- 
plied to several codes, among them: cycle codes, turbo-like codes and RA codes IIFK04llHE05l 
IGBTTH . LDPC codes [|FMS+07[lDDKW08[[KV06llADS09llHETT]| . and expander codes llFSOSl 
ISkalll . Our work is motivated by the problem of finite-length and average-case analysis of suc- 
cessful LP-decoding of Tanner codes. There are very few works on this problem, and they deal 
only with specific cases. For example, Feldman and Stein HFSOSII analyzed special expander 
codes, and Goldenberg and Burshtein HGBllll deal with repeat- accumulate codes. 



Previous results. Combinatorial characterizations of sufficient conditions for successful de- 
coding are based on so called "certificates". That is, given a channel observation y and a 
codeword x, we are interested in a one-sided error test that answers the questions: is x optimal 
with respect to y? is it unique? Note that the test may answer "no" for a positive instance. A 
positive answer for such a test is called a certificate for the optimality of a codeword. Upper 
bounds on the word error probability are obtained by lower bounds on the probability that a 
certificate exists. 

Koetter and Vontobel [IKV06II analyzed LP decoding of regular LDPC codes. Their analy- 
sis is based on decomposing each codeword (and pseudocodeword) to a finite set of minimal 
structured trees (i.e., skinny trees) with uniform vertex weights. Arora et al. [IADS09II extended 
the work in [IKV06II by introducing nonuniform weights to the vertices in the skinny trees, and 
defining local-optimality . For a BSC, Arora et al. proved that local optimality implies both 
ML-optimality and LP-optimality. They presented an analysis technique that performs a finite- 
length density evolution of a min-sum process to prove bounds on the probability of a decoding 
error. Arora et al. also pointed out that it is possible to design a re-weighted version of the 
min-sum decoder for regular codes that finds the locally-optimal codeword if such exists for 
trees whose height is at most quarter of the girth. This work was further extended in HHEllll 
to memoryless channels. The analyses presented in these works [IKV061 |ADS09[ iHEl 11 are 
limited to skinny trees, the height of which is bounded by a half of the girth of the Tanner 
graph. 

Vontobel HVonlOII extended the decomposition of a codeword (and pseudocodeword) to 
skinny trees in graph covers. This enabled Vontobel to mitigate the limitation on the height by 
the girth. The decomposition is obtained by a random walk, and applies also to irregular Tanner 
graphs. 

Various iterative message-passing decoding algorithms are derived from the belief propaga- 
tion algorithm (e.g., max-product IIWLK95II . attenuated max-product HFKOOL tree-reweighted 
belief-propagation HWJWOSll . etc.). The convergence of these BP-based iterative decoding al- 
gorithms to an optimum solution has been studied extensively in various settings (see e.g., 
IIWLK951 IFKOOI IWFOli ICFOll lCDE+05[ IWJW051 IRUOTI IJPTTH ). However, bounds on the 
time and message complexity of these algorithms are not considered. The analyses in these 
works often rely on the existence of a single optimal solution in addition to other conditions 
such as: single-loop graphs, large girth, large reweighting coefficients, consistency conditions, 
etc. 

Jian and Pfister HJPllll analyzed a special case of the attenuated max-product decoder HFKOOII . 
for regular LDPC codes. They considered skinny trees in the computation tree, the height of 
which is greater than the girth of the Tanner graph. Using contraction properties and consis- 
tency conditions, they proved sufficient conditions under which the message-passing decoder 



converges to a locally optimal codeword. This convergence also implies convergence to the 
LP-optimum and therefore to the ML-codeword. 

While local-optimality characterizations were investigated for the case of finite-length anal- 
ysis of regular LDPC codes, no characterizations have been presented for the general case of 
Tanner codes. This paper presents the first decoding algorithm for finite-length LDPC codes 
over MBIOS channels with bounded time complexity that combines two properties: (1) it is a 
message-passing algorithm, and (2) the algorithm has an ML-certificate. 

Contributions. We present a new combinatorial characterization of local-optimality for ir- 
regular Tanner codes with respect to any memoryless binary-input output-symmetric (MBIOS) 
channel (Definition S]). Local optimality is characterized via costs of deviations based on sub- 
trees in computation trees of the Tanner graph. Consider a computation tree with height 2h 
rooted at some variable node. A deviation is based on a subtree such that (i) the degree of a 
variable node equals to its degree in the computation tree, and (ii) the degree of a local-code 
node equals some constant d ^ 2, provided that d is at most the minimum distance of the 
local-codes. Furthermore, level weights w E R'ji are assigned to the levels of the tree. Hence, 
a deviation is a combinatorial structure that has three main parameters: deviation height h, 
deviation level weights w E R'|i, and deviation "degree" d. Therefore, the new definition of 
local-optimality is based on three parameters: h Efi,w E R^, and d ^ 2. 

This characterization extends the notion of deviations in local-optimality in four ways: 
(i) no restrictions are applied to the degrees of the nodes in the Tanner graph, (ii) arbitrary 
local linear codes may be associated with constraint nodes, (iii) deviations are subtrees in the 
computation tree and no limitation is set on the height of the deviations; in particular, their 
height may exceed the girth of the Tanner graph, (iv) minimal deviations may have a degree 
d ^ 2 in the check nodes (as opposed to skinny trees in previous analyses), provided that 
d is at most the minimum distance of the local-codes. We prove that local-optimality in this 
characterization implies ML-optimality (Theorem |5]). We utilize the equivalence of graph cover 
decoding and LP-decoding for Tanner codes, implied by Vontobel and Koetter IIVK05II . to prove 
that local-optimality suffices also for LP-optimality (Theorem [TOl). We present an efficient dy- 
namic programming algorithm that computes a local-optimality certificate for a codeword with 
respect to a given channel output (Algorithm [T]). 

We present a new message-passing iterative decoding algorithm, called normalized weighted 
min-sum (NWMS) algorithm (Algorithm O. NWMS algorithm applies to irregular Tanner codes 
with single parity-check local-codes (e.g., LDPC codes and HDPC codes). The characteriza- 
tion of local-optimality for Tanner codes with single parity-check local-codes has two param- 
eters: (i) a certificate depth h, and (ii) a vector of layer weights w E R'ji \ {O'^}. (Note that 
the local codes are single parity checks, and therefore the deviation degree d equals 2.) We 



prove that, for any finite h, the NWMS decoder is guaranteed to compute the ML codeword in 
h iterations if an /i-locally-optimal codeword exists (Theorem [T6l). Furthermore, the output of 
NWMS can be efficiently certified. The number of iterations, h, may exceed the girth. Because 
local-optimality is a pure combinatorial property, the results do not rely on convergence. The 
time and message complexity of NWMS is 0{\E\ ■ h) where |-E| is the number of edges in the 
Tanner graph. Local optimality, as defined in this paper, is a sufficient condition for successful 
decoding by our BP-based algorithm in loopy graphs. 

Previous bounds on the probability that a local-optimality certificate exists [|ADS09[|HE11II 
hold for regular LDPC codes. The same bounds hold also for successful decoding by NWMS. 
These bounds are based on proving that a local-optimality certificate exists with high probabil- 
ity for noise thresholds close to the BP threshold. Specifically, noise thresholds of p* ^ 0.05 
in the case of BSC flADS09L and a* ^ 0.735 in the case of BLAWGN channel HHEUH are 
proven. 

Finally, because trees in our new characterization may have degrees bigger than two, they 
contain more vertices. Hence this characterization leads to improved bounds for successful 
decoding of regular Tanner codes (Theorems [27] and[38]) . These bounds extend the probabilistic 
analysis of the min-sum process by Arora et al. flADS09ll to a sum-min-sum process on regular 
trees. For regular Tanner codes, we prove bounds on the word error probability of LP-decoding 
under MBIOS channels that are inverse doubly-exponential in the girth of the Tanner graph. 
We also prove bounds on the threshold of regular Tanner codes whose Tanner graphs have 
logarithmic girth. This means that if the noise in the channel is below that threshold, then 
the decoding error diminishes as a function of the block length. Note that Tanner graphs with 
logarithmic girth can be constructed explicitly (see e.g., IIGal63ll '). 

To summarize, our contribution is threefold, (i) We present a new combinatorial charac- 
terization of local-optimality for Tanner codes with respect to any memoryless binary-input 
output symmetric (MBIOS) channel. This characterization provides an ML-certificate and an 
LP-certificate for a given codeword. The certificate can be efficiently computed by a dynamic 
programming algorithm. Based on this new characterization, we present two applications of 
local-optimality. (ii) A new efficient message-passing decoding algorithm, called normalized 
weighted min-sum (NWMS), for irregular Tanner codes with single parity-check local codes 
(e.g., LDPC codes and HDPC codes). The NWMS algorithm is guaranteed to find the locally 
optimal codeword in h iterations, where h determines the height of the local-optimality cer- 
tificate. Note that h is not bounded and may be larger than the girth of the Tanner graph (i.e., 
decoding with local-optimality guarantee "beyond" the girth), (iii) New bounds on the word 
error probability are proved for LP-decoding of regular Tanner codes. 



Organization. The remainder of this paper is organized as follows. Section |2]provides back- 
ground on ML-decoding and LP-decoding of Tanner codes over MBIOS channels. Section [3] 
presents combinatorial certificate that applies to ML-decoding for codewords of Tanner codes. 
In Section HI we prove that the certificate applies also to LP-decoding for codewords of Tan- 
ner codes. In Section [51 we present an efficient certification algorithm for local-optimality. In 
Section [6l we prove a structural decomposition for codewords of Tanner codes used as a key 
element in the proof of the main theorem of Section [3l Section |7] presents the NWMS iterative 
decoding algorithm for irregular Tanner codes with single parity-check local-codes, followed 
by a proof that NWMS finds the locally-optimal codeword. In Section [8l we use the combi- 
natorial characterization of local-optimality to bound the error probability of LP decoding for 
regular Tanner codes. Finally, conclusions and discussion are given in Section |9l 

2 Preliminaries 

Graph Terminology. Let G = {V, E) denote an undirected graph. Let Mciv) denote the set 
of neighbors of node v ^V, and for a set 5 C V let MciS) = Ut,G5-^G'(^)- ^^^ "^^SgI^) — 
\J^g{v) I denote the edge degree of node v in graph G. A path p = (t>, . . . , u) in G is a sequence 
of vertices such that there exists an edge between every two consecutive nodes in the sequence 
p. Let s{p) denote the first vertex of path p, and let t(p) denote the last vertex of path p. If 
s{p) = t{p) then the path is closed. A simple path is a path with no repeated vertex. A simple 
cycle is a closed path where the only repeated vertex is the first and last vertex. A path p is 
backtrackless if every two consecutive edges along p do not close a cycle. Let \p\ denote the 
length of a path p, i.e., the number of edges in p. Let ^^(r, t>) denote the distance (i.e., length of 
a shortest path) between nodes r and v in G, and let girth (G) denote the length of the shortest 
cycle in G. Letp and q denote two paths in a graph G such that t{p) = s{q). The path obtained 
by concatenating the paths p and q is denoted hy p o q. 

An induced subgraph is a subgraph obtained by deleting a set of vertices. The subgraph of 
G induced by S (^V consists of S and all edges in E, both endpoints of which are contained 
in S. Let Gs denote the subgraph of G induced by S. 

Tanner-codes and Tanner graph representation. Let G = {V U J,E) denote an edge- 
labeled bipartite-graph, where V = {f i, . . . , f at} is a set of A^ vertices called variable nodes, 
and J = {Gi, . . . , Gj} is a set of J vertices called local-code nodes. We denote the degree of 

Gjhyrij. 

Let C = {C : C is an [rij, kj, dj] code, j E [J]} denote a set of J linear local-codes. 
The local code C'' corresponds to the vertex Gj G J. We say that Vi participates in C'' if {vi, Gj) 
is an edge in E. The edges incident to each local-code node Gj are labeled {1, . . . ,nj}. This 



labeling specifies the index of a variable nodes in the corresponding local-code. 

A word X = {xi, . . . , xat) G {0, 1}^ is an assignment to variable nodes in V where Xj is 
assigned to t>j. Let Vj denote the set MdCj) ordered according to labels of edges incident 
to Cj. Denote by xy^ G {0, l}*^^ the projection of the word x = (xi, . . . , xn) onto entries 
associated with Vj. 

The Tanner code C{G, C ) based on the labeled Tanner graph G is the set of vectors x G 
{0, 1}^ such that xy is a codeword in C for every j G \J]. 

Let dj denote the minimum distance of the local code C'' . The minimum local distance d* 
of a Tanner code C{G, C ) is defined by d* = min^ dj. We assume that d* > 2. 

If the bipartite graph is (d^, d^j) -regular, i.e., the vertices in V have degree dL and the 
vertices in J have degree dn, then the graph defines a (rf^, dji)-regular Tanner code. 

If the Tanner graph is sparse, i.e., \E\ = 0{N), then it defines a low-density Tanner code. 
A parity code is the code that contains all binary words with even Hamming weight. Tanner 
codes with parity local codes that are based on sparse Tanner graphs are called low-density 
parity-check (LDPC) codes. 

Consider a Tanner code C{G, C ). We say that a word x = {xi, ..., xjy) satisfies local-code 
C if its projection xy^ is in C . The set of words x that satisfy the local-code C is denoted by 
C^, i.e., C^ = {x G {0, 1}^ : xy^ G C^}. Namely, the resulting code C^ is the extension of the 
local-code C from length rij to length N . The Tanner code is simply the intersection of the 
extensions of the local codes, i.e., 

C(G,C'^)= f|C^\ (1) 

LP decoding of Tanner codes over memoryless channels. Let Cj G {0, 1} denote the 
zth transmitted binary symbol (channel input), and let yi G R denote the ith received sym- 
bol (channel output). A memoryless binary-input output- symmetric (MBIOS) channel is de- 
fined by a conditional probability density function f{yi\ci = a) for a G {0, 1}, that satisfies 
f{yi\0) = f{—yi\l). The binary erasure channel (BBC), binary symmetric channel (BSC) 
and binary-input additive white Gaussian noise (BI-AWGN) channel are examples for MBIOS 
channels. In MBIOS channels, the log-likelihood ratio (LLR) vector A G R^ is defined by 
Xi{yi) = In { ifll'Zi) ) for every input bit i. For a code C, Maximum-Likelihood (ML) decoding 
is equivalent to 

x'^'^d/) = argmin(A(y),x), (2) 

a;6conv(C) 

where conv(C) denotes the convex hull of the set C. 

In general, solving the optimization problem in Q for linear codes is intractable IIBMvT78ll . 
Feldman et al. [|Fel03[ IFWK05II introduced a linear programming relaxation for the problem 



of ML decoding of Tanner codes with single parity check codes acting as local codes. This 
definition is based on a fundamental polytope that corresponds to the Tanner graph G. We 
consider an extension of this definition to the case in which the local codes are arbitrary as 
follows. The I 
is defined by 



follows. The generalized fundamental polytope V = 'P(G', C ) of a Tanner code C = C{G, C 



V= f] conv(C^). (3) 

Note that a Tanner code may have multiple representations by a Tanner graph and local codes. 
Moreover, different representations {G, C ) of the same Tanner code C may yield different 
generalized fundamental polytopes V{G, C ) . If the degree of each local-code node is constant, 
then the generalized fundamental polytope can be represented hy 0{N + \J'\) variables and 
OdJT"!) constraints. Typically, \J'\ = 0{N), and the generalized fundamental polytope has 
an efficient representation. Such Tanner codes are often called generalized low-density parity- 
check codes. 

Given an LLR vector A for a received word y, LP-decoding is defined by the following 
linear program: 

x"(?/) = argmin (A(|/), x). (4) 



The difference between ML-decoding and LP-decoding is that the fundamental polytope 
V{G, C ) may strictly contain the convex hull of C. Vertices of V{G, C ) that are not code- 
words of C must have fractional components and are called pseudocodewords. 



3 A Combinatorial Certificate for an ML Codeword 

In this section we present combinatorial certificates for codewords of Tanner codes that apply 
both to ML-decoding and LP-decoding. A certificate is a proof that a given codeword is the 
unique solution of maximum-likelihood decoding and linear-programming decoding. The cer- 
tificate is based on combinatorial weighted structures in the Tanner graph, referred to as local 
configurations. These local configurations generalize the minimal configurations (skinny trees) 
presented by Vontobel HVonlOU as extension to Arora et al. flADS09ll . We note that for Tanner 
codes, the support of each weighted local configuration is not necessarily a local valid configu- 
ration. For a given codeword, the certificate is computed by a dynamic-programming algorithm 
on the Tanner graph of the code (see Section[5]). 

Notation: Let y E R" denote the word received from the channel. Let \ = \{y) denote the 
LLR vector for y. Let G = {V U J', E) denote a Tanner graph, and let C{G) denote a Tanner 
code based on G with minimum local distance d*. Let x E C{G) be a candidate for x^^^(y) 
andx^^{y). 



Definition 1 (Path-Prefix Tree). Consider a graph G = (V, E) and a node r G V. Let V denote 
the set of all backtrackless paths in G with length at most h that start at node r, and let 

E = {ipi,P2) ^ V X V \ pi is a prefix of p2, bi| + 1 = b2|}- 

We identify the empty path in V with (r). Denote by T,^{G) = {V , E) the patli-prefix tree ofG 
rooted at node r with height h. 

Patli prefix trees of G that are rooted in variable nodes are often called computation trees. 
We consider also path prefix trees of subgraphs that may be either rooted at a variable node or 
at a local-code node. 

We use the following notation. Because vertices in T^{G) are paths in G, we denote vertices 
in path-prefix trees by p and q. Vertices in G are denoted by m, f , r. For a path p eV , let s{p) 
denote the first vertex {source) of path p, and let t{p) denote the last vertex {target) of path p. 
Denote by Prefix^ (p) the set of proper prefixes of the path p, i.e., 

Prefix^ (p) = |g | g is a prefix of p, 1 ^|5'|< \p\]- 

Consider a Tanner graph G = {VU J,E) and let T^{G) = {V , E) denote a path-prefix tree of 
G. Let V = {p I p G V, t{p) e V}, and J = {p\p eV, t{p) e J}. Paths in V are called 
variable paths , and paths in J' are called local-code paths. 

The following definitions expand the combinatorial notion of minimal valid deviations IIWib96ll 
and weighted minimal local-deviations (skinny trees) [|ADS09[ IVonlOl to the case of Tanner 
codes. 

Definition 2 {d-tree). Consider a Tanner graph G = (V U J.,E). Denote by 'T^^{G) = 
(V U J , E) the path-prefix tree ofG rooted at node r eV. A subtree T C T^^{G) is a rf-tree 
if: (i) T rooted at (r), (ii) for every local-code path p E T H J', deg-j-^p) = d, and (Hi) for 
every variable path p G T n V, deg^(p) = deg^-ah (p). 

Note that the leaves of a rf-tree are variable paths because a d-tree is rooted in a variable 
node and has an even height. Let T[r, 2h, d] (G) denote the set of all d-trces rooted at r that are 
subtrees of 7;2^(G). 

In the following definition we use "level" weights w = {wi, . . . , Wh) that are assigned to 
variables paths in a subtree of a path-prefix tree of height 2h. 

Definition 3 (tu- weighted subtree). Let T = (V U jf , -E) denote a subtree ofT^^{G), and let 
w = {wi, . . . ,Wh) G Mil denote a non-negative weight vector Let wj- : V — ;■ R denote a 
weight function based on weight vector w for variable paths p E V defined as follows. Ifp is 



an empty variable path, then wrip) = 0- Otherwise, 

where i = [^] . We refer to wj- as a w-weighed subtree. 

For any iw- weighted subtree wj- of 7^'^{G), let tig,t,w : V — ?■ R denote a function whose 
values correspond to the projection of w-j- to the Tanner graph G. That is, for every variable 
node V in G, 

TTG,T,wi^) - XI ^r(p)- (6) 

{per:t{p)=v} 

We remark that: (i) If no variable path in T ends in v, then 'n'G,T,wiv) = 0. (ii) If h < 
girth(G)/4, then every node v is an endpoint of at most one variable path in T^'^{G), and the 
projection is trivial. However, we deal with arbitrary heights h, and the projection deals with 
many variable paths that end at the same variable node v. (iii) 7CG,T,wiv) E [0, 1] for every 
weight vector w E E^ \ {0^}. 

For a Tanner code C{G), let B^^ C [0,1]^ denote the set of all projections of w- weighted 
d-trees to G. That is. 



^r = {^c,r,. I '^ e U T[r,2h,d]{G)}. 



rev 

Vectors in Bj^ are referred to as deviations. 

For two vectors x E {0, 1}^ and / E [0, 1]^, let x © / E [0, 1]^ denote the relative 
point defined by (x © /)« = |xj — fi\ nFelOBH . The following definition is an extension of 
local-optimality nADSOQiFVonlOU to Tanner codes on memory less channels. 

Definition 4 (local-optimality). Let C{G) C {0, 1}^ denote a Tanner code with minimum 
local distance d*. Let w E IR^\{0^} denote a non-negative weight vector of length h and let 
2 ^ d ^ d*. A codeword x E C{G) is {h, w, d)-\oca\\y optimal with respect to A G R^ if for 
all vectors (5 E B^ , 

{X,x®f3)>{X,x). (7) 

Based on random walks on the Tanner graph, Vontobel showed that (h,w, 2)-\ocal opti- 
mality is sufficient both for ML-optimality and LP-optimality. The random walks are defined 
in terms derived from the generalized fundamental polytope. We extend the results of Vonto- 
bel nVonlOl to "thicker" sub-trees by using probabilistic combinatorial arguments on graphs 
and the properties of graph cover decoding HVKOSII . Specifically, we prove that {h, w, d)-\oca\ 
optimality, for any 2 ^ d ^ d*, implies both ML- and LP-optimality (Theorems 15] and FTOl). 

10 



Given the decomposition of Lemma [TT] proved in Section[6l the following theorem is obtained 
by modification of the proof of IIADS091 Theorem 2] or HHElli Theorem 6]. 

Theorem 5 (local-optimality is sufficient for ML). Let C{G) denote a Tanner code with mini- 
mum local distance d*. Let h be some positive integer and w = {wi, . . . , Wh) G R^ denote a 
non-negative weight vector Let X G R^ denote the LLR vector received from the channel. If 
X is an {h, w, d)-locally optimal codeword w.r.t. A and some 2 ^ d ^ d*, then x is also the 
unique maximum-likelihood codeword w.r.t. X. 

Proof. We use the decomposition proved in Section [6] to show that for every codeword x' ^ x, 

{X,x') > (A,x). Let z = X ® x'. By linearity z G C{G). Moreover, 2; 7^ 0^ because x ^ x'. 
Because d* ^ 2, it follows that ||2;||i ^ 2. By Lemma [TT] there exists a distribution over the set 
B^J"\ such that E, „(„,)/? = ^. Let a = ^ < 1. Let / : [0, 11^ ^ R be the affine linear 
function defined by f{/3) ^{X,x®/3) = (A, x) + T.^ii-'^Y'^'^A- Then, 

(A, x) < E (») (A, X (B l3) (by local-optimality of x) 

= (A, X © E (,„)/?) (by linearity of / and linearity of expectation) 

= (A, X © az) (by Lemma [TTT) 

= (A, (1 — a)x + a{x © z)) 

= (A, (1 — a)x + ax') 

= {1 — a){X,x) + a{X,x'). 

which implies that (A, x') > (A, x) as desired. D 

4 Local Optimality Implies LP-Optimality 

In order to prove a sufficient condition for LP optimality, we consider graph cover decoding 
introduced by Vontobel and Koetter nVKOSH . We note that the characterization of graph cover 
decoding and its connection to LP decoding can be extended to the case of Tanner codes in the 
generalized setting. 

We use the terms and notation of Vontobel and Koetter HVKOSII in the statements of Propo- 
sition|6]and Lemma|9l Specifically, let G denote an M-cover of G. Let x = x^*^ G C{G) and 
A = A'''^'^ G R^'^^ denote the M-lifts of x and A, respectively. 

In this section we consider the following setting. Let C{G) denote a Tanner code with 
minimum local distance d*. Let w G R^\{0^} for some positive integer h and let 2 ^ rf ^ d*. 

Proposition 6 (local-optimality of all-zero codeword is preserved by M-lifts). O'^ is an{h,w,d)- 
locally optimal codeword for X G R^ if and only ifO^''^ is an {h, w, d)-locally optimal code- 
word w.r.t. X. 
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Proof. Consider the surjection ip of d-trees in the path-prefix tree of G to (i-trees in the path- 
prefix tree of G. This surjection is based on the covering map between G and G. Given a 
deviation /3 = vr^j -y-^, let /3 = tig,v{T),w The proposition follows because (A, /3) = (A, /3). D 

For two vectors y, z G R^, let "*" denote coordinatewise multiplication, i.e., y * z = 
{yi ■ Zi, . . . ,yN ■ zn). For a word x G {0, 1}^, let b G {±1}^ denote a vector defined by 

Proposition 7. For every A G R^ an J every f3 G [0,1]^, 

(6*A,/3) = (A,x©/3)-(A,x). (8) 

Proof. For /3 G [0, 1]^, it holds that (A, x © /3) = (A, x) + E«=i(-1)^''^*A- Hence, 

N 



(A,x©/3)-(A,x) = 5^(-l)^»A,A 



«=i 



= (&*A,/3) 

D 

The following proposition states that the mapping (x, A) t-^ (0^,6 * A) preserves local 
optimality. 

Proposition 8 (symmetry of local-optimality). For every x E C, x is {h, w, d)-locally optimal 
for A if and only ifO'^ is {h, w, d)-locally optimal for b * X. 

Proof By Proposition |71 (A, x ® l3) - (A, x) = {b*X,l3). D 

The following lemma states that local-optimality is preserved by lifting to an M-cover. 

Lemma 9. x is {h, w, d)-locally optimal w.r.t. A if and only if x is {h, w, d)-locally optimal 
w.r.t. A. 

Proof. Assume that x is {h, w, (i)-locally optimal codeword for A. By Proposition [8l 0^'^^ is 
{h, w, (i)-locally optimal w.r.t. (— 1)^* A. By Proposition[6l 0^ is (h, w, d)-\oca\\y optimal w.r.t. 
(6 * A). By Proposition [H x is {h, w, (i)-locally optimal w.r.t. A. Each of these implications is 
necessary and sufficient, and the lemma follows. D 

The following theorem is obtained as a corollary of Theorem [5] and Lemma [9l The proof 
is based on arguments utilizing properties of graph cover decoding. Those arguments are used 
for a reduction from ML-optimality to LP-optimality similar to the reduction presented in the 
proof of HHElll Theorem 8]. 

Theorem 10 (local optimality is sufficient for LP optimality). Ifx is an {h, w, d)-locally opti- 
mal codeword w.r.t. A, then x is also the unique optimal LP solution given A. 
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5 Verifying Local Optimality 

In this section we address the problem of how to verify whether a codeword x is {h, w, d)- 
locally optimal with respect to A. By Proposition [H this is equivalent to verifying whether 0^ 
is (/i, U7, (i)-locally optimal with respect to 6 * A, where hi = (— 1)"^'. 

The verification algorithm is listed as Algorithm[TJ It applies dynamic programming to find, 
for every variable node v, a d-tree %, rooted at v, that minimizes the cost {b * A, 'Kg,Tv,w)- The 
algorithm returns false if and only if it finds a deviation with nonpositive cost. 

The algorithm is presented as a message passing algorithm. In every step, a node propagates 
to its parent the minimum cost of the d-subtree that hangs from it based on the minimum values 
received from its children. The time and message complexity of Algorithm [T] is Od-E"! • h), 
where E denotes the edge set of the Tanner graph. 

The following notation is used in Line 8 of the algorithm. For a set S of real values, let 
minWlS} denote the ith smallest member in S. 

Algorithm 1 verify-lo(x, A, /;,, i(;,(i) - An iterative verification algorithm. Let G = (V U 
J , E) denote a Tanner graph. Given an LLR vector A G R.'^', a codeword x E C{G), level 
weights w E R^, and a degree d E 1N+, outputs "true" if x is (h, w, (i)-locally optimal w.r.t. A; 
otherwise, outputs ''false". 



1 


Initialize: Wv eV: X'^^ K- (-1)^" 


2 

3 


VC Ej,yvE M{C): /i[7i^„ ^ 
for / = to /i - 1 do 


4 
5 
6 


for sA\veV,C E U{v) do 

,,(0 . «'h-i \/ 1 1 Y^ ,,('-!) 

end for 


7 
8 
9 


for sA\C Ej,vE U{C) do 

/^2L ^ ih ■ Eti minW {/iit^ : v' E M{C) \ {v}] 
end for 


10 


end for 


11 

12 
13 
14 


for all t; G V do 

v^ (h-l) 

if /it, ^ then {min-cost w-weighted d-tree rooted at v has non-positive value} 
return false; 


15 


end if 


16 


end for 


17 


return true; 



6 Constructing Codewords from Weighted Trees Projections 

In this section we prove Lemma [TTl the key structural lemma in the proof of Theorem [51 This 
Lemma states that every codeword of a Tanner code is a finite sum of projections of weighted 
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trees in the computation trees of G. 

Throughout this section, let C{G) denote a Tanner code with minimum local distance d*, 
let X denote a nonzero codeword, let h denote some positive integer, and let w E R\ \ {0^} 
denote level weights. 

Lemma 11. Consider a codeword x ^ 0^. Then, for every 2 ^ d ^ d*, there exists a 
distribution p over d-trees ofG of height 2h such that for every weight vector w G R^ \ {0}, it 
holds that 

X = ||a;||i ■ Ep[7iG,r,w]- 

The proof of Lemma [TT] is based on lemmas [T2lfT3] and corollary [141 Lemma [T2l states that 
every codeword x E C{G) can be decomposed into a set of weighted path-prefix trees. The 
number of trees in the decomposition equals ||x||i. Lemma [T3] states that every weighted path- 
prefix tree is a convex combination of weighted d-trees. This lemma implies that the projection 
of a weighted path-prefix tree equals to the expectation of projections of weighted d-trees. 

For a codeword x E C{G) C {0, 1}^, let Vx — {v \ x^ = 1}. Let Gx denote the subgraph 
of the Tanner graph G induced by Vx U MiVx)- 

Lemma 12. For every codeword x ^ 0^, for every weight vector w E R'ji, and for every 
variable node v eV, 

Proof \i Xy = 0, then 7^G,T^'^{Gx),wi''j) = 0- It remains to show the equality holds for variable 
nodes v eVx- 

Consider an all-one weight vector r] = 1^. Construct a path-suffix tree rooted at v. Level £ 
of this path-suffix trees consists of all backtrackless paths in Gx of length i that end at node v 
(see Figure [U). We denote this level by Pi{v). 

We claim that for every v E Vx and 1 ^ £ ^ 2h, 



E^r-(P) = ^- (9) 



"(p) h 

The proof is by induction on L The induction basis, for £ = 1, holds because |-Pi(f)| = 
deg(^(i;) and r]^2h (p) = ^^J- ,, for every p E Pi{v). The induction step is proven as follows. 
For each p E Pt{v), let aug(p) = {q E Pi^i{v) | p is a suffix of g}. Note that |aug(p)| = 
degg^ {s{p)) — I. Moreover, for each q E aug(p), 

^rf, (?) 1 

"'"' — = (10) 

r/7-2.^(p) dega^ {s{p)) - 1 
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\v = t{p) 



• ••^sIpT 



'P(-i{v) 
'Pi{v) 



Figure 1: Set of all backtrackless paths Pe{v) as augmentation of the set P^_i(f ) as viewed by 
the path-suffix tree of height i rooted at v, in proof of Lemma [TTl 



Hence, 

Q6aug{p) 

Finally, Pi^i{v) is the disjoint union of [j{aug{p) \ p E Peiv)}- It follows that 



E ^r-)(^)= E ^r-)(P)- 



(11) 



qGPe+i{v) 



pePtiv) 



By the induction hypothesis we conclude that J2neP M ^r^'" (^) ~ V^' ^^ required. Note 
that the sum of weights induced by r] on each level is 1/h, both for levels of paths beginning 
in variable nodes and in local-code nodes. In the rest of the proof we focus only on even levels 
that start at variable nodes. We now claim that 



Yl ^rfXp) 



peP2t{v) 



kill 



(12) 



Indeed, by Definition [3] it holds that w^rih (p) = r]n-2h (p) ■ -n^ ■ h for every p E P2i{v). 

s{p) s{p) 11*^111 

Therefore, Equation (fT2l) follows from Equation ^. 
The lemma follows because for every v E Vx, 

h 



r-GVj; 



h 



E 



We 

Ml 



1. 



D 
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Lemma 13. Consider a subgraph Gx of a Tanner graph G, where x G C{G) \ {0^}. Then, 
for every variable node r G G^, every positive integer h, every 2 ^ d ^ d*, and every weight 
vector w G M!^, it holds that 

^VHG.) = Ep. [wt] 

where pr is the uniform distribution over T[r, 2h, d]{Gx)- 

Proof. Let G^ = (V^ U J7^, E^) and let Wj-2h(^Q_^^ denote a w- weighted path-prefix tree rooted at 
node r with height 2h. We claim that the expectation of w-weighted d-tree wr G T[r, 2h, d] (Gx) 
equals Wj-2h(^Q^) ifwj- is chosen uniformly at random. 

A random rf-tree in T[r, 2h, d]{Gx) is obtained as follows. Start from the root r. For each 
variable path, take all its augmentations, and for each local-code path choose d — 1 distinct 
augmentations uniformly at random. Let T G T[r, 2h, d](Gx) denote such a random rf-tree, 
and consider a variable path p G T^^{Gx)- Then, 

Prrenr,.,MoAP^r)= n deg/(t"(g))-l - ^^^^ 

Note the following two observations: (i) ifp^T, then wr{p) = 0, and (ii) if p G T, then 
the value ofwrip) is constant and does not depend on T. Moreover, from the two observations 
above we have 

Erer[r,2?»,d](G.) [wt{p)] = wt{p) ■ Prrer[r,2/i,d](G.)(P e T). (14) 

By Definition [3l for p G T we have 

/ X ^ W\p\/2 I 1 y-r 1 



By substituting Equations (|T3I) and (|T5l) in Equation (|T4l) . we conclude that 
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Corollary 14. For every positive integer h, every 2 ^ d ^ d*, and every weight vector w G R^, 
it holds that 



T^G,T?i^{G:,),w = Ep, [tTg.T.w;] • 



16 



Proof. By definition of t^g,t?^{g^),w^ we have 

T^G,T?HG.),wiv) = Yl '^T,^HG.)iP)- (16) 

By Lemma O and linearity of expectation we have 

Yl '^VHgM = Y ^T€T[r,2h,cl]{G.)[wT 

{peVHG.y.t(p)=v} {p&VHG.y.t{p)=v} 



E. 



TGT[r,2h,d](G:,) 



Yl ^^(^) 

^{peV'{G^):t{p)=v} 



(17) 



Now, for variable paths p that are not in a rf-tree T, wr{p) = 0. Hence, if a d-tree T is a 
subtree of 7^^{Gx), then 

Y ^T^p^ = Y ^^(^) 

{pG7?'^(G,):t(p)=i>} {per:t(p)=i)} 

- TrG,T,M. (18) 



From Equations (fT6l)- (|T8l) we conclude that for every t> G V, 

'^G,%^''(G,),w{v) = Erer[r,2/i,d](G,)[7rG,r,^ 
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Before proving Lemma[TTl we state a proposition from probability theory. 
Proposition 15. Let {pr}fLi denote K probability distributions. Let p = j^ Ylir=i P'- Then, 

K 

r=l 

Proof of Lemma [77] By Lemmas [I2] and Corollary [14] we have for every v &Vx 

Xv=Y '^G,T?h^G^)A'") (19) 

reVx 

= Y ^p- [^G,r,«-] • 



reVa: 



Let p denote the distribution defined by p = -vrAr- -J^rev^ Pr ■ ^y Proposition[T5]and Equation[T9l 

Xy = \\x\\i ■ Ef,[nG,T,w], 
17 



and the lemma follows. D 

7 Message-Passing Decoding with ML Guarantee for Irreg- 
ular LDPC Codes 

In this section we present a weighted min-sum decoder (called, NWMS) for irregular LDPC 
codes over any memoryless binary-input output- symmetric channels. In Section [TT] we prove 
that the decoder computes the maximum-likelihood (ML) codeword if a locally-optimal code- 
word exists (Theorem [T6l). Note that Algorithm NWMS is not presented as a min-sum process. 
However, in Section mi an equivalent min-sum version is presented. 

We deal with Tanner codes based on Tanner graphs G = {V U ^7, E} with single parity- 
check local-codes. Local-code nodes C E J^ in this case are called check nodes. The graph G 
may be either regular or irregular. All the results in this section hold for every Tanner graph, 
regardless of its girth, degrees, or density. 

A huge number of works deal with message-passing decoding. We point out three works 
that can be viewed as precursors to our decoding algorithm. Gallager [|Gal63ll presented the 
sum-product iterative decoding algorithm for LDPC codes. Tanner UTanSlll viewed itera- 
tive decoding algorithms as message passing algorithms over the edges of the Tanner graph. 
Wiberg HWib96ll characterized decoding failures of the min-sum iterative decoding algorithm 
by negative cost trees. Message-passing decoding algorithms proceed by iterations of "ping- 
pong" messages between the variables nodes and the local-code nodes in the Tanner graph. 
These messages are sent only along the edges. 

Algorithm description. Algorithm NWMS (A, h, w), listed as Algorithmic is a normalized un- 
weighted version of the min-sum algorithm for decoding Tanner codes with single parity-check 
local-codes. The input to algorithm NWMS consists of an LLR vector A G R^, an integer h > 
that determines the number of iterations, and a nonnegative weight vector w E R^. For each 
edge (f , C), each iteration consists of one message from the variable node v to the check node 
C (that is, the "ping" message), and one message from C to f (that is, the "pong" message). 
Hence, the time and message complexity of Algorithm [2] is Od-Ej • h). 

Let iJ'l_^c denote the "ping" message from a variable node f G V to an adjacent check- 
node C E J in iteration / of the algorithm. Similarly, let fij^^ denotes the "pong" message 
from C G ^7 to t> G V in iteration /. Denote by /i^ the final value computed by variable node 
V E V. Note that NWMS does not add wqXv in the computation of ^^ in Line 1 1 for ease of 
presentatiorlll. The output of the algorithm x E {0, 1}^ is computed locally by each variable 



'Adding wqXv to /i„ in Line 1 1 requires changing the definition of deviations so that they also include the root 
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node in Line 12. 

Algorithm NWMS may be applied to any memoryless binary-input output- symmetric chan- 
nel (e.g., BEC, BSC, AWGN, etc.) because the input is the LLR vector. 

Algorithm 2 nwms(A, h, w) - An iterative normalized weighted min-sum decoding algorithm. 
Given an LLR vector A G R^ and level weights w E R'j:, outputs a binary string x E {0,1}^ . 

Initialize: VC E J,yv E M{C) : /i^T^J, ^ 
for Z = to /i — 1 do 

for sA\veV,C E U{v) do {"PING"} 

^v^C ^ Acgai-") "" dcgG(t;)-l Z-^C'(iN(v)\{C} f^C'-^v 

end for 

for aWC Ej,vE U{C) do {"PONG"} 

7: /^cL ^ f ]\u&M{C)\{v} sign(/^ilc) J " min„e^(c)\M { l/^i'l^cl } 

end for 
end for 

for all y G V do {Decision} 

if /i„ > 0, 

1 otherwise. 
13: end for 

The following theorem states that nvv^ms(A, /i, w) computes an (/i,w, 2)— locally-optimal 
codeword w.r.t. A if such a codeword exists. Hence, Theorem [T6]provides a sufficient condition 
for successful iterative decoding for any finite number h of iterations. In particular, the number 
of iterations may exceed the girth. Theorem [16] implies an alternative proof of the uniqueness 
of an {h,w, 2)-locally optimal codeword that is proved in Theorem [51 The proof appears in 
Section [7H 



9 

10 
11 



12: X„ ^ ' ' " ' 



Theorem 16 (NWMS finds the locally optimal codeword). Let G = (yuj',E) denote a Tanner 
graph and let C{G) C {0, 1}^ denote the corresponding Tanner code with single parity-check 
local-codes. Let h E IN+ and let w E R'j: \ {O'^} denote a non-negative weight vector Let 
A G R^ denote the LLR vector of the channel output. If x E C{G) is an {h,w, 2) -locally 
optimal codeword w.r.t. A, then NWMS(A, /i, w) outputs x. 

The NWMS decoding algorithm has an ML-certificate. Namely, if a locally-optimal code- 
word exists, the dynamic programming algorithm verify-lo described in Section [5] can be 
used to verify whether NWMS(A, /i, w) outputs an (/i, w, 2)-locally optimal codeword. If so, 
then, by Theorem[51 the output of NWMS (A, h, w) is the unique ML-codeword. 



of each d-tree. 
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Corollary [26] states that for MBIOS channels, the probability that NWMS fails is independent 
of the transmitted codeword. Hence, the following corollary is a contra-positive of Theorem [T6] 
provided the all-zero codeword assumption. 

Corollary 17. Assume that the all-zero codeword is transmitted, and let A G R^ denote the log- 
likelihood ratio for the received word. IfNWMS{X, h, w) fails to decode the all-zero codeword 
for w E R+\{0^}, then there exists a deviation j3 G B2 such that (A, /3) ^ 0. 

Hence, for a fixed h and w G R^\{0^}, 

Pr{NWMS(A,/i,w)fails|c = 0^} ^ Pr{3/3 G Sf ^ such that (A, /3) ^0|c = 0^}. (20) 

Bounds on the existence of a non-positive weighted deviation (i.e., the right-hand side in Equa- 
tion (I20I) ') are discussed in Section 1911 

7.1 Proof of Theorem [16] - NWMS Finds the Locally Optimal Codeword 

Proof outline. The proof of Theorem [16] is based on two observations. (1) We present an 
equivalent algorithm, called NWMS2 (Section [7. 1.1 1) , and prove that Algorithm NWMS2 outputs 
the all-zero codeword if 0^ is locally optimal (Sections I7.1.2H7.13]) . (2) In Lemma [25] we 
prove that algorithm NWMS is symmetric (Section 17.1.41) . This symmetry is with respect to 
the mapping of a pair (x, A) of a codeword and an LLR vector to a pair (0^, A°) of the all- 
zero codeword and a corresponding LLR vector A° = (— 1)"^ * A (recall that "*" denotes a 
coordinate- wise vector multiplication). 

To prove Theorem [T6l we prove the contrapositive statement, that is, if x 7^ NWMS (A, h, w), 
then X is not {h, w, 2)-locally optimal w.r.t. A. Let x denote a codeword, and define b G {±1}^ 
by h = (— 1)^\ Define A° = 6 * A. By definition A = 6 * A°. The proof is obtained by the 
following derivations: 

X y^ NWMS{X, h,w) 
^x ^ X ® NWMS(A°, h, w) [Lemma [25| symmetry of NWMS] 

^O^^NWMS(A°,/i,u;) 

^3/3 G -B5"'Ma°, 13) ^0 [Lemma[22l local optimality (LO)] 

■x^x is not {h, w, 2)— locally optimal for A. [Prop. [8] symmetry of LO] 



Note that the only implication that is only sufficient is in the 4th line, which is based on 
Lemma [22] 
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D 
We now prove the two lemmas used in the foregoing proof. 

7.1.1 NWMS2 : An Equivalent Version 

Algorithm NWMS is input the log-likelihood ratio. We refer to this algorithm as a min-sum 
algorithm in light of the general description of Wiberg IIWib96ll in the log-domain. In Wiberg's 
description, every check node finds a minimum value from a set of functions on the incom- 
ing messages, and every variable node computes the sum of the incoming messages and its 
corresponding channel observation. Hence the name "min-sum". 

Let y G R^ denote channel observations. For a E {0, 1}, define the log-likelihood of yi by 
Aj(a) = — log (Pr(?/j|cj = a)) . Note that the log-likelihood ratio Aj for yi equals A,j(l) — Aj(0). 

Algorithm NWMS2(A(0), A(l), /i, w), listed as Algorithm [3l is a normalized w-weighted 
min-sum algorithm. Algorithm NWMS2 computes separate "reliabilities" for "0" and "1". 
Namely, fil'_^^{a) and fij^^ia) denote the messages corresponding to the assumption that node 
V is assigned the value a (for a E {0, 1}). The higher the values of these messages, the lower 
the likelihood of the event Xy = a. 

The main difference between the presentations of Algorithm[2]and Algorithm [3] is in Line 7. 
Consider a check node C and valid assignment x E {0, Ij'^'^sCC) ^q variable nodes adjacent to C 
with even weight. For every such assignment x in which x^ = a, the check node C computes 
the sum of the incoming messages fij_^(j{xu) from the neighboring nodes u E J^{C) \ {v}. 
The message iJ,Q_^^{a) equals the minimum value over these valid summations. 

Following Wiberg [|Wib96l Appendix A. 3], we claim that Algorithms[2land[3]are equivalent. 

Claim 18. Let A, A(0), and A(l) in R^ denote the LLR vector and the two log-likelihood 
vectors for a channel output y E R^. Then, for every h E 1N+ and w E R^, the following 
equalities hold: 

1. liy\c = 1^v\M - /^l'ic(O) ^'"^ l^c^v = /^cL.(l) - /^c^t,(0) in every iteration I. 

2. /i„ = /it,(l) - /it,(0). Hence NWMS(A,/i, w) and NWMS2(A(0), A(l), /i, w) output the 
same vector x. 

7.1.2 NWMS2 as a Dynamic Programming Algorithm 

In Lemma [19] we prove that Algorithm NWMS2 is a dynamic programming algorithm that 
computes, for every variable node v, two min-weight valid configurations. One configuration 
is 0-rooted and the other configuration is 1-rooted. Algorithm NWMS2 decides a;^, = if the 
min-weight valid configuration rooted at v is 0-rooted, otherwise decides x^ = 1. We now 
elaborate on the definition of valid configurations and their weight. 
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Algorithm 3 NWMS2(A(0), A(l), h, w) - An iterative normalized weighted min-sum decoding 
algorithm. Given an log-likelihood vectors A(a) G R^ for a E {0, 1} and level weights 
w E R'ji, outputs a binary string x E {0, 1}^. 



10 
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Initialize: VC E J,^v E Af{C), Va E {0, 1} : fi't^l{a) ^ 
for / = to /i - 1 do 
for a\\vEV,C E Afiv), a E {0, 1} do {"PING"} 

end for 

for allC Ej,v E U{C), a G {0, 1} do {"PONG"} 

end for 
end for 

for allv eV do {Decision} 

if (/i,(l)-/i,(0)) >0, 



12: a;,, ^ . 

1 otherwise 

13: end for 



Valid configurations and their weight. Fix a variable node r eV. We refer to r as the root. 
Consider the path-prefix tree 7^^{G) rooted at r consisting of all the paths of length at most 2h 
starting at r. Denote the vertices of 7^^ by V U ^7, where paths inV = {p \ p E V, t{p) E V} 
are variable paths, and paths vn J = {p\p eV , t{p) E J} are parity-check paths. Denote by 
(r) the empty path, i.e., the path consisting of only the root r. 

A binary word z E {0, l}'^' is interpreted as an assignment to variable paths p eV where 
Zp is assigned to p. We say that 2; is a valid configuration if it satisfies all parity-check paths in 
J . Namely, for every check path q E J, the assignment to its neighbors has an even number 
of ones. We denote the set of valid configurations of T^^' by config(7^^''). 

The weight W-j-zh {z) of a valid configuration z is defined by weights yV^2h (p) that are 
assigned to variable paths p G V as follows. We start with "level" weights w = (wi, . . . , Wh) E 
R^ that are assigned to levels of variable paths in T^'^. Define the weight of a variable path 
p E V with respect to w b) 



W\p\/2 y-r 1 



>^.»W^:,:^^- n 



There is a difference between Definition [3] and W-j-'^h^p). In Definition [3] the product is taken 
over all paths in Prefix^ (p) . However, in Wr^h {p) the product is taken only over variable paths 

in Prefix"^ {p) . 



^We use the same notation as in Definition[3] 
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The weight of a valid configuration z G {0, 1}'^' is defined by 

peV\{(r)} 

Given a variable node r E V and a bit a G {0, 1}, our goal is to compute the value of a 
min- weight valid configuration >V™™(r, a) defined by 

W""°(7;'^ a) = argmm{WT^H{z) \ z G config(7;''^), Z(,) = a}. (21) 

In the following lemma we show that NWMS2 computes >V™™(7^'*, a) for every r G V 
and a G {0, 1}. The proof is based on interpreting NWMS2 as a dynamic programming. See 
Appendix lAl for details. 

Lemma 19. Consider an execution o/NWMS2(A(0), A(l), h, w). For every variable node r, 

From Line 12 in Algorithm NWMS2 we obtain the following corollary that characterizes 
NWMS2 as a computation of min- weight configurations. 

Corollary 20. Let x denote the output of NWMS2(A(0), A(l), h, w). For every variable node 

r, 

'o if >v°^'°(7;2\ 1) > >v'"'°(7;^\ 0), 

1 otherwise. 
Define the W* cost of a configuration z in T^^'* by 

Note that W^2h(-2) uses the LLR vector A (i.e., A„ = A^(l) - A^(0)). 

Corollary 21. Let x denote the output of NWMS(A, h, w). Let z* denote a valid configuration 
in Tj^^ with minimum W* cost. Then, x^ = zJy 



X. 



r 



Proof The derivation in Equation (|22l) shows that the valid configuration z* that minimizes the 
W* cost also minimizes the W cost. 

argmin >Vr^2h(2) = argmin {WT2h(z) - Wr^2h(0l^l)} (22) 

valid ^€7?'" '^ valid z&T?*^ 



(J} 

valid 



argmin | ^ Ai(p)(l) ■ Wr?^{p) - ^ Xt^Q) ■ Wr?u{p) \ 

alid z&T?^ I - , ,-, -, J 



pGV:2p = l pGV:Zp = l 

= argmin ^ Aj(p) ■ Wr^h{p) ■ Zp = argmin W!^2h{z). 

valid ^67?^ « "^ valid zGT?'^ 

peV 
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Equality (1) relies on the fact that >V7-2h(0'^') is a constant. The elements Xt(p){zp) ■ yVj-2h{p) 
in W'j2h{z) where Zp = are reduced by the substraction of the same elements in W7-2h(0l^l). 
Therefore, leaving in Equality (2) only elements that correspond to bits Zp = 1. Equality (3) is 
obtained by the LLR definition Af(p) = Af(p)(l) — Af(p)(0). 

Let £ = NWMS(A,/i,w) and y = NWMS2(A(0), A(l), /i, ty). By Corollary [201 and Equa- 
tion l22l ijr = zJ^. By Claim[T8l Xr = Vr, and the corollary follows. D 

7.1.3 Connections to Local Optimality 

The following lemma states that the NWMS algorithm computes the all-zero codeword if 0^ is 
locally optimal. 

Lemma 22. Let x denote the output o/NWMS(A, h, w). IfO^ is {h, w, 2)-locally optimal w.r.t. 
A, then x = 0^. 

Lemma 23. Let x denote the output o/nwms(A, h, w). Ifx^ = 1, then there exists a deviation 
l3 G B2 corresponding to a w-weighted 2-tree such that (A, /3) ^ 0. 

Proof. We prove the contrapositive statement. Assume that x ^ 0^ . Hence, there exists a 
variable node v for which x^ = 1. Consider T^'^ = {V U J', E). Then, by Corollary [2T1 there 
exists a valid configuration z* E {0, l}'^' in T^^'^ with zf^ = 1 that satisfies 

Vvalid configuration y G T^^. W*^2h (z*) ^ W*^2h (y) . (23) 

Let T{z*) denote the subgraph of %^^ induced by V{z*) U A/'(V(z*)) where V{z*) = {p e 
V\z* = 1}. Note that T{z*) is a forest. Because zJ. = 1 and z* is a valid configuration, 
the forest T{z*) must contain a 2-tree of height 2h rooted at the node v; denote this tree by 
T. Let r G {0, l}'^' denote the support of T, and let z'^ G {0, l}'^' denote the support of 
T{z*) \ T . Then, 2* = r + 2;'^, where z^ is also necessarily a valid configuration. By linearity 
and disjointness of r and z°, we have 

y^r^n {z*) = W*r2H (r + 2°) = W*r2H (r) + Wf 2. (/). (24) 

Because 2;° is a valid configuration, by Equation (|23l) . we have Wtihiz*) ^ WX-2hiz^)- By 
Equation ^, W^idr) ^ 0. 

Let w* G -R'^' denote the vector whose component indexed by p G V equals yVf2h(p) ■ Tp. 
The vector w* equals to the w-weighted 2-tree wr according to Definition [3l Hence, (3 = 
TiG,T,w e -BJ""^ satisfies (A, /3) = W^^hir) < 0. We therefore conclude that 0^ is not (/i, w, 2)- 
locally optimal w.r.t. A and the lemma follows. D 
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7.1.4 Symmetry and the All-Zero Codeword Assumption 

We define symmetric decoding algorithms (see I1RU081 Definition 4.81] for a discussion of 
symmetry in message passing algorithms). 

Definition 24 (symmetry of decoding algorithm). Let x E C denote a codeword and let h E 
{±1}^ denote a vector defined by bi = (—1)^*. Let A denote an LLR vector A decoding 
algorithm, DEC(A), is symmetric with respect to code C, if 

\/x EC. X® DEC(A) = DEC(6 * A). (25) 

The following lemma states that NWMS algorithm is symmetric. The proof is by induction on 
the number of iterations. 

Lemma 25 (symmetry of nwms). Fix h E IN+ and w E R+. Consider A E E^ and a 
codeword x E C{G). Let b E {±1}^ denote a vector defined by bi = (—1)^'. Then, 

X©NWMS(A,/l,w) = NWMS(6* X,h,w). (26) 

Proof. Let /iJ,^.c[A] denote the message sent from f to C in iteration / given an input A. Let 
Aic'_^i,[A] denote the corresponding message from C to v. From the decision of NWMS in Line 
12, it's sufficient to prove that /i^'l^P [A] = (-1)^" ■/iJ,'2,c[6*A] and fi'§^^[\] = (-l)^"-/i2L[^* 
A] for every ^ I ^ h — I. 

The proof is by induction on /. The induction basis, for / = —1, holds because /i^„[A] = 
(—1)^" ■ /i(^^& * A] = for every codeword x. 

The induction step is proven as follows. By induction hypothesis we have 



^•^^ ^ taGV y C"eAf{v)\{C} 



C'(^Af{v)\{C} 



:-ir-f^Zc[b*M- 
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For check to variable messages we have by induction hypothesis, 

= ( n ^ig<(-ir-/^i'^c[&*A]))- nun {|(-l)-.^^^^^^ 

^ueAf{C)\{v} ^ 

= ( n (-ir)-/^cL[&*A]. 

^ u€j\f{C)\{v} ^ 

Because x is codeword, for every single parity check C we have nuGA^(c)\{»;| (~1)^" = (— 1)^"- 
Therefore, /ic'_^^[A] = (— 1)"^" ■ ^q^^^ * A] and the claim follows. D 

The following corollary follows from Lemma [25] and the symmetry of an MBIOS channel. 

Corollary 26 (All-zero codeword assumption). Fix h E 1N+ and w E R^. For MBIOS chan- 
nels, the probability that NWMS fails is independent of the transmitted codeword. That is, 

PrJNWMS decoding fails} = Pr{NWMS(A, /i, w) ^ 0^|c = 0^}. 

Proof. Following Lemma [25l for every codeword x, 

Pr{NWMS(A, h,w) ^ x\c = x} = Pr{NWMS(6 * A, /i, w) 7^ 0^ | c = x}. 

For MBIOS channels, Pr(Ai | 0) = Pr(-Ai | 1). Therefore, the mapping (x, A) ^ (0^, h*\) 
preserves probability measure. We apply this mapping to (x, 6 * A) 1— )■ (0^, h * h * \) and 
conclude that 

Pr{NWMS(6 * A, /i, w) ^ 0^ I c = x} = Pr{NWMS(A, h,w)^Q^ \c = 0^}. 

D 

8 Bounds on the Error Probability of LP-Decoding Using 
Local-Optimality 

In this section we analyze the probability that a local optimality certificate for regular Tanner 
codes exists, and therefore LP decoding succeeds. The analysis is based on the study of a "sum- 
min-sum" process that characterizes rf-trees of a regular Tanner graph. We prove upper bounds 
on the error probability of LP decoding of regular Tanner codes in memory less channels. The 
upper bounds on the error probability imply lower bounds on the threshold of LP decoding. 
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We apply the analysis to the binary symmetric channel, and compare our results with previous 
results on expander codes. The analysis presented in this section generalizes the probabilistic 
analysis of Arora et al. [IADS09II from 2-trees (skinny trees) to d-trees for any d ^ 2. 

In the remainder of this section, we restrict our discussion to (c^l, dij) -regular Tanner codes 
with minimum local distance d*. Let d denote a parameter such that 2 ^ d ^ d*. 

Theorem [27] summarizes the main results presented in this section for binary symmetric 
channels, and generalizes to any MBIOS channel as described in Section [83l Concrete bounds 
are given for a (2, 16)-regular Tanner code with code rate at least 0.375 when using [16, 11, 4]- 
extended Hamming codes as local codes. 

Theorem 27. Let G denote a {d^, dji) -regular bipartite graph with girth g, and letC{G) denote 
a Tanner code based on G with minimum local distance d*. Let x G C{G) be a codeword. 
Suppose that y E {0,1}^ is obtained from x by flipping every bit independently with probability 
p. Then, 

1. [finite length bound] Let d = do, p ^ po, {di, dfj) = (2, 16), and d* = 4. For the values 
of da and pq in Table Ujfl it holds that x is the unique optimal solution to the LP decoder 
with probability at least 



Pr(x^P(y) = x) ^1- N -a 



(d-i) LisJ 



for some constant a < 1. 



2. [asymptotic bound] Let d = d^, {dL,dji) = (2,16), d* = 4, and g = r2(logA^) suffi- 
ciently large. For the values of do and po in Table\l]p it holds that x is the unique optimal 
solution to the LP decoder with probability at least 1 — exp{—N^) for some constant 
< 6 < 1, provided that p ^ Poido). 

3. Let d' = d — 1, d'j^ = di — 1, and d' = dji — 1. For any {di, dfj) and 2 ^ d ^ d* 
s.t. d'l^ ■ d' ^ 2, the codeword x is the unique optimal solution to the LP decoder with 
probability at least 1 — N ■ a^'^'^'^'^ " for some constant a < 1, provided that 

mm I {ai{p,d,dL,dR,t)) ■ {a2{p,d,dL,dR,t)Y ^' "^ f < ^' 



where 

d'-i 



fc=0 ^ ^ ^k=d' ^ ^ ^ 

a2ip,d,dL,dR,t) = f f j((l-p)e-*+pe*)'^'. 
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do 


Po 


"finite" 


3 


0.0086 


4 


0.0218 


"asymptotic" 


3 


0.019 


4 


0.044 



Table 1 : Computed values of po for finite do < d* in Theorem [27J Values are presented for 
(2, 16)-Tanner code with rate at least 0.375 when using [16, 11, 4] -extended Hamming codes as 
local codes, (a) finite-length bound: Vp ^ po bound on the word error probability that is inverse 
doubly-exponential in the girth of the Tanner graph, (b) asymptotic -bound: For g = f2(log N) 
sufficiently large, LP decoder succeeds w.p. at least l—ex]9(—iV'') for some constant < 5 < I, 
provided that p ^ Poido). 



Proof Outline. Theorem 1271 follows from Lemma 1301 Lemma 1331 Corollary [361 and Corol 
lary [37] as follows. The first part, that states a finite-length result, follows from Lemma 
and Corollaries [36l and [371 bv taking s = < h < igirth(G') which holds for any Tanner 
graph G. The second part, that deals with an asymptotic result, follows from Lemma [30[ and 
Corollaries [36] and [37] by fixing s = 10 and taking g = r2(log A^) sufficiently large such that 
s < h = 6(logA^) < igirth(G'). It therefore provides a lower bound on the threshold of 
LP-decoding. The third part, that states a finite-length result for any (di, (i/j)-regular LDPC 
code, follows from Lemma [30l and Lemma [33l D 



We refer the reader to Section 19.21 for a discussion on the results stated in Theorem [ 
We now provide more details and prove the lemmas and corollaries used in the proof of Theo- 
rem [27] 

In order to simplify the probabilistic analysis of algorithms for decoding linear codes over 
symmetric channels, we apply the assumption that the all-zero codeword is transmitted, i.e., 
c = 0^. Note that the correctness of the all-zero assumption depends on the employed de- 
coding algorithm. Although this assumption is trivial for ML decoding because of the sym- 
metry of a linear code C{G), it is not immediately clear in the context of LP-decoding. Feld- 
man et al. IIFel03l[FWK05ll noticed that the fundamental polytope V{G) of Tanner codes with 
single parity-check local codes is highly symmetric, and proved that for binary-input output- 
symmetric channels, the probability that the LP decoder fails is independent of the transmitted 
codeword. The symmetry property of the polytope remains also for the generalized fundamen- 
tal polytope of Tanner codes based on non-trivial linear local codes. Therefore, one can assume 
that c = 0^ when analyzing LP-decoding failure for linear Tanner codes. The following corol- 
lary is the contrapositive statement of Theorem [10] given c = 0^. 
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Corollary 28. For every fixed he'n,w e B.\\{<d^}, and2^d^ d*, 

Pr{LP decoding fails} ^ Pr{3/3 G B^^^ such that (A, /3) ^ 0|c = 0^}. 

8.1 Bounding Processes on Trees 

Let G be a (rf^, (iij)-regular Tanner graph, and fix /i < igirth(G'). Let T^^I'iG) denote the path- 
prefix tree rooted at a variable node vq with height 2h. Since h < igirth(G), it follows that 
the projection of 7^^{G) to G is a tree. We direct the edges of T^^ so that it is an in-branching 
directed toward the root Vq (i.e., paths from all nodes are directed toward the root t>o). For 
/ G {0,...,2/i}, denote by Vi the set of vertices of T^^ at height / (the leaves have height and 
the root has height 2h). Let r C V{T^^) denote the vertex set of a (i-tree rooted at vq. 

Definition 29 ((/i, cj, (i)-Process on a (d^, (iR)-Tree). Let u E R'j, denote a weight vector Let 
A denote an assignment of real values to the variable nodes ofTv^, we define the u-weighted 
value of a d-tree r by 

h-l 

val^{T; A) = ^ ^ ui- Xy. 

1=0 vGTnV2i 

Namely, the sum of the values of variable nodes in r weighted according to their height. 
Given a probability distribution over assignments X, we are interested in the probability 

nA,d,di,dfl(/i,cu) = PrA< min val^{T] A) ^ L (27) 

[ TeT[vo,2h,d] j 

In other words, nA,d,d^,d^ (h, to) is the probability that the minimum value over all d-trees of 
height 2h rooted in some variable node Vq in a {di, c//j) -bipartite graph G is non-positive. For 
every two roots vq and vi the trees T^J^' and T^^'^ are isomorphic, hence nA,d,d^,d^(/i, cu) does 
not depend on the root vq. 

With this notation, the following lemma connects between the {h, co, (i)-process on (d^, d^)- 
trees and the event where the all-zero codeword is (/i, ty, (i)-locally optimal. We apply a union 
bound utilizing Corollary |2ll as follows. 

Lemma 30. Let G be a {di, dnj-regular bipartite graph andw G R'j: \ {O'^} be a weight vector 
with h < |girth(G). Assume that the all-zero codeword is transmitted, and let X G R^ denote 
the LLR vector received from the channel. Then, 0^ is {h, w, d)-locally optimal w.r.t. X with 
probability at least 

l-N- Ilx44^,d^{h, u), where Ui = Wh-i ■ dl^ ■ (d^ - 1)'-^+^ ■ {d - lf-\ 

and with at least the same probability, 0^ is also the unique optimal LP solution given X. 
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Note the two different weight notations that we use for consistency with [IADS09II : (i) w 
denotes weight vector in the context of (/i, w^ (i)-local optimality certificate, and (ii) oj denotes 
weight vector in the context of d-trees in the (/i, w, (i)-process. A one-to-one correspondence 
between these two vectors is given by ui = Wh-i ■ dj^^ ■ {di — ly-^+i . (c? — 1)'^^' for ^ / < /i. 
From this point on, we will use only u in this section. 

Following Lemma [301 it is sufficient to estimate the probability nA,d,d^,d^(/i, u) for a given 
weight vectors, a distribution of a random vector A, constant 2 ^ d ^ d*, and degrees {di^dR). 
Arora et al. IIADS09II introduced a recursion for estimating and bounding the probability of the 
existence of a 2-tree (skinny tree) with non-positive value in a [h, uj, 2)-process. We generalize 
the recursion and its analysis to rf-trees with 2 ^ d ^ d*. 

For a set S of real values, let mint^^lS} denote the ith smallest member in S. Let {7} 
denote an ensemble of i.i.d. random variables. Define random variables Xq, . . . ,Xh^i and 
Yq, . . . , Yh-i with the following recursion: 

Yo = Uo7 (28) 

Xi = ^minW{l^('\...,l^(''^^')} {0^1 <h) (29) 

Yi = ua + xi!_\ + ... + xi^\-'^ {o<i<h) (30) 

The notation X^^\ . . . , X^''^ and Y^^\ . . . , Y^''^ denotes k mutually independent copies of the 
random variables X and Y, respectively. Each instance of Yj, ^ / < h, uses an independent 
instance of a random variable 7. Note that for every ^ I < h, the d—l order statistic random 
variables | min^^^lF/ ' , . . . ,Yi^ '} \ l^i^rf— l}in Equation (|29l) are dependent. 

Consider a directed tree T = 7I,o of height 2h, rooted at node vq. Associate variable nodes 
of T at height 2/ with copies of Yj, and check nodes at height 21 + 1 with copies of Xi, for 
^ I < h. Note that any realization of the random variables {7} to variable nodes in T can be 
viewed as an assignment A. Thus, the minimum value of a d-tree of T equals ^j^^ -^h-v This 
implies that the recursion in (I28l)-(l30l) defines a dynamic programming algorithm for comput- 
ing minT-g7-[^g 2/i,d] val^(T; A). Now, let the components of the LLR vector A be i.i.d. random 
variables distributed identically to {7}, then 

nw„.,(/.,a;) = Pr|^X«, ^ o|. (31) 

Given a distribution of {7} and a finite "height" h, the challenge is to compute the distri- 
bution of Xi and Yi according to the recursion in (l28l)- (|30l) . The following two lemmas play a 
major role in proving bounds on Ti-x^d^L^Ri^^ ^)- 
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Lemma 31 ( nADS09ll ). For every t ^ 0, 

Let d' = d — \, d'l^ = di — 1 and d'j^ = d^ — 1. 
Lemma 32 (following [IADS09II ). For0^s<l<h,we have 



fc=0 



k 



Proof. See Appendix |Bl D 

In the following subsection we present concrete bounds on Il\^d,dL,diti^,(^) for the BSC. 
The bounds are based on Lemmas [3T] and [32l The technique used to derive concrete bounds 
for the BSC may be applied to other MBIOS channels. For example, concrete bounds for the 
BLAWGN channel can be derived by a generalization of the analysis presented in HHEUL 

8.2 Analysis for Binary Symmetric Channel 

Consider the binary symmetric channel with crossover probability p denoted by BSC(p). In 
the case that the all-zero codeword is transmitted, the channel input is q = for every i. 
Hence, Pr(Ai = -log{^)) =]9, andPr(Ai = +log{^)) = l-p. Since Ux,d,d^4^{h,uj) is 
invariant under positive scaling of the vector A, we consider in the following analysis the scaled 
vector A in which Aj = +1 w.p. p, and —1 w.p. (1 — p). 

Following the analysis of Arora et al. nADS09L we apply a simple analysis in the case of 
uniform weight vector u. Then, we present improved bounds by using a non-uniform weight 
vector. 

8.2.1 Uniform Weights 

Consider the case where u = 1^. Let ai = Ee^*''^*' and 02 — (^?)(Ee~*'^)'^' where 7 *~' Aj, 
and define a = miut^Q ai ■ a^ ^ ■ Note that a\ ^ 0:2 (see Equation (l37])).We consider the 
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case where a < 1. By substituting notations of ol\ and 0:2 in Lemma l32l for s = 0, we have 



«! -^ ■ «2 



fc=0 

(4.d')'-i 
1 w-^^')' 



By Lemma [3n we conclude that 



To analyze parameters for which nA,d,d^,d^(/i, l'^) — )■ 0, we need to compute a\ and 0:2 as 
functions of p, d, d^ and dji. Note that 

jd' -2k w.p. (t^)/(l - p)'^'«-^ Vfc. ^ fc < d', 
JCq ^ < , (32) 

[-d' w.p. E'^d'(t)/(i-p)'«-'- 

Therefore, 



ai(p,rf,d,,rfH,t) = J^f'^'Ap^l-pY'^'n-^^e-'^'^'-''^ (33) 

fc=0 ^ ^ 

a2{p,d,dL,dn,t) = (^Jf\{il-p)e-'+peY- (34) 

The above calculations give the following bound on Ilx,d,dL,dRi^^ 1^)- 

Lemma 33. Let p G (0, |) and let d, d^, d^ ^ 2 s.t. d'j^ ■ d' ^ 2. Denote by ai and a^ the 
functions defined in d35])-d34l). and let 

a = min<^ [ai{p,d,dL,dji,t)) ■ {a2{p,d,dL,dii,t)) ^ 



32 



Then, for /i G IN and u = 1^, we have 

^\,d,dL,dR{h,uj) ^ a 



dL-d'j -dL 



Note that if a < 1, then nA,d,d^,d^(/i, l'*) decreases doubly-exponentially as a function of h. 
For (2, 16)-regular graphs and d E {3, 4}, we obtain the following corollary. 

Corollary 34. Let d^ = % and d^ = 16. 

1. Let d = 3 andp ^ 0.0067. Then, there exists a constant a < 1 such that for every /i G IN 

and w = l^ 

2. Let d = A andp ^ 0.0165. Then, there exists a constant a < 1 such that for every /i G IN 

and w = l^ 

The bound on p for which Corollary [34] applies grows with d. This fact confirms that 
analysis based on denser trees, i.e., rf-trees with d > 2 instead of skinny trees, implies better 
bounds on the error probability and higher lower bounds on the threshold. Also, for d > 2, we 
may apply the analysis to (2, ci/j) -regular codes; a case that is not applicable by the analysis of 
Ammetal. [IADS09L 

8.2.2 Improved Bounds Using Non-Uniform Weights 

The following lemma implies an improved bound for Ilx^d,dL-dR{h,(^) using a non-uniform 
weight vector uj. 

Lemma 35. Let p E (0, |) and let d,dL,dR^ 2 s.t. d'^-d' ^ 2. For s G IN and a weight vector 

UJ G R+, let 

a = min {Ee-^^} • (('^)) (2V^(r^)'^') '^^. (35) 

Let Lo^^^ G R'ji denote the concatenation of the vector cJ G R^ and the vector (p, . . . , p) G 
R^~*. Then, for every h > s there exists a constant p > such that 



d'r,\ ,_ ^— T^dA ^T^^ 



L 



nw„,,(/i,u;(^))^(^(^^fj(2V^(r^) I 

Proof. See Appendix |C] D 
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s 


PO 




s 


PO 





0.0086 


4 


0.0164 


1 


0.011 


5 


0.0171 


2 


0.0139 


6 


0.0177 


3 


0.0154 


10 


0.0192 



Table 2: Computed values of po for finite s in Corollary|36l Values are presented for (t/^, dfj) = 
(2, 16) and d = 3. 

Consider a weight vector cJ with components cJi = ((di — l){d — 1))'. This weight vector 
has the effect that every level in a skinny tree r contributes equally to valjj^r; |A|) (note that 
|A| = 1). For h > s, consider a weight vector co'^^^ E R'ji defined by 



CO 



(p) 



LOi if ^ / < s, 
p ii s ^ I < h. 



Note that the first s components of o;^''^ are geometric while the other components are uniform. 
For a given p, d, di, and dji, and for a concrete value s we can compute the distribution of 
Xg using the recursion in (I28l)-(l30l). Moreover, we can also compute the value min(j.o Ee~*^^ . 
For (2, 16)-regular graphs we obtain the following corollaries. Corollary [36] is stated for the 
case where (i = 3, and Corollary [37] is stated for the case where d = A. 

Corollary 36. Let p ^ pq, d = 3, t/^ = 2, and d^ = 16. For the following values ofpo and s 
in Table^it holds that there exists constants p > and a < 1 such that for every h > s, 



-1 



U,,,,,^,,^ih,u^P^)^—-{pil-p))- -a 
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Corollary 37. Let p ^ pq, d = 4, di = 2, and da = 16. For the following values ofpo and s 
in Table\3\it holds that there exists constants p > and a < 1 such that for every h > s, 



U,^,,,„,,{h,u^^^)^-{p{l-p)) 



4 3'' 



Note that for a fixed s, the probability Ilx,d,dL,dR{h, oj) decreases doubly-exponentially as a 
function of h. 



8.3 Analysis for MBIOS Channels 

Theorem [27] generalizes to MBIOS channels as follows. 
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s 


PO 




s 


PO 





0.0218 


4 


0.039 


1 


0.0305 


5 


0.0405 


2 


0.0351 


6 


0.0415 


3 


0.0375 


10 


0.044 



Table 3: Computed values of po for finite s in Corollary |37l Values are presented for (t/^, dfj) = 
(2, 16) and d = A. 

Theorem 38. Let G denote a {di^dRj -regular bipartite graph with girth VL{\ogN), and let 
C{G) C {0, 1}^ denote a Tanner code based on G with minimum local distance d*. Consider 
an MBIOS channel, and let A G R^ denote the LLR vector received from the channel given 
c = 0^. Let 7 G R- denote a random variable independent and identically distributed to 
components of \. Then, for any {di^dji) and 2 ^ d ^ d* s.t. (rfj^ — l)((i — 1) ^ 2, LP -decoding 
succeeds with probability at least 1 — exp(— A^'^)/or some constant < 5 < 1, provided that 



mm 

i>0 



^d-l 



Ee" 



-tXa 



(1) 



where Xq = X]j=i niin'*'{7 
and distributed identically to 7. 



d- 

,7 



(Ee''y M 1x1. 



(dR 1) j ^^j ^/jg random variables 7*^*-' are independent 



9 Conclusions and Discussion 

We present a new combinatorial characterization of local-optimality for Tanner codes with 
respect to any MBIOS channel. This characterization provides an ML-certificate and an LP- 
certificate for a given codeword. Moreover, the certificate can be efficiently computed by a 
dynamic programming algorithm. Two applications of local-optimality are presented based on 
this new characterization, (i) A new message passing decoding algorithm for irregular LDPC 
codes, called NWMS. The NWMS algorithm is guaranteed to find the locally optimal codeword 
if one exists, (ii) Bounds for LP-decoding failure are proved in the case of regular Tanner codes. 
We discuss these two applications of local-optimality in the following subsections. 

9.1 Applying NWMS Algorithm to Regular LDPC Codes 

The NWMS algorithm is a generalization of the min-sum algorithm (a.k.a. max-product in the 
probability-domain) and other BP-based decoding algorithms in the following sense. When 
restricted to regular Tanner graphs and exponential level weights (to cancel the normalization 
in the variable node degrees), the nwms algorithm reduces to the standard min-sum algo- 
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rithm [|WLK95[ IWib96ll . Reductions of NWMS to other BP-based algorithms (see e.g., atten- 
uated max-product HFKOOH and normalized BP-based [ICF02[ lCDE+0511 ) can be obtained by 
other weight level functions. 

Many works on the BP-based decoding algorithms study the convergence of message pass- 
ing algorithms (e.g., IIWFOli IWJW051 IRUOll IJP11H ). In particular, bounds on the time and 
message complexity are not considered. The analyses in these works rely on the existence of a 
single optimal solution in addition to other conditions such as: a single cycle, large girth, large 
reweighing coefficient, consistency conditions, etc. On the other hand, the NWMS algorithm is 
guaranteed to compute the ML codeword within h iterations if a locally optimal certificate of 
depth h exists for some codeword. Moreover, the certificate can be computed efficiently (see 

Algorithm [T])- 

In previous works [|ADS09[|HE11II . the probability that a locally optimal certificate of depth 
h exists for some codeword was investigated for regular LDPC codes with h < igirth(G). 
Consider (c/^, cZ/j) -regular LDPC code whose Tanner graph G has logarithmic girth, let h < 
|girth(G) and define a constant weight vector w = 1^. In that case, the message normalization 
by variable node degrees has the effect that each level of variable nodes in a 2-tree contributes 
equally to the cost of the w-weighted value of the 2-tree. Hence, the set B2 of deviations 
equals to the set of (rfj^ — 1) -exponentially weighted skinny trees [|ADS09[|HE11II . Following 
Equation (l20l) . we conclude that the previous bounds on the probability that a locally optimal 
certificate exists IIADS09irHEllll apply also to the probability of NWMS decoding success. 

Consider (3, 6) -regular LDPC codes whose Tanner graphs G have logarithmic girth, and 
let h = igirth(G') and w = l'^. Then, NWMS (A, h, w) succeeds in recovering the transmitted 
codeword with probability at least 1 — exp(— A^"^) for some constant < 5 < 1 in the following 
cases: (1) In a BSC with crossover probability p < 0.05 (implied by IIADS091 Theorem 5]). 
(2) In a BLAWGN channel with ^ ^ 2.67dB (implied by IIHETTI Theorem 1]). 

It remains to explore good weighting schemes (choice of vectors w) for specific families of 
irregular LDPC codes, and prove that a locally optimal codeword exists with high probability 
provided that the noise is bounded. Such a result would imply that NWMS decoding algorithm 
is a good efficient replacement for LP-decoding. 

9.2 Bounds on the Word Error Probability for LP-Decoding of Tanner 
Codes 

In Section [8] we proved bounds on the word error probability of LP-decoding of regular Tanner 
codes. In particular, we considered a concrete example of (2, 16)-regular Tanner codes with 
[16, 11, 4]-Hamming codes as local-codes and Tanner graphs with logarithmic girth. The rate 
of such codes is at least 0.375. For the case of a BSC with crossover probability p, we prove 
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a lower bound of p* = 0.044 on the noise threshold. Below that threshold the word error 
probability decreases doubly exponential in the girth of the Tanner graph. 

Most of the research on the error correction of Tanner codes deals with families of expander 
Tanner codes. How do the bounds presented in Section [8] compare with results on expander 
Tanner codes? The error correction capability of expander codes depends on the expansion, 
thus a fairly large degree and huge block-lengths are required to achieve good error correction. 
Our example for which results are stated in Theorem [27lfl) and UJ\ 2) relies only on a 16- 
regular graph with logarithmic girth. Sipser and Spielman I1SS96II studied Tanner codes based 
on expander graphs and analyzed a simple bit-flipping iterative decoding algorithm. Their 
novel scheme was later improved, and it was shown that expander Tanner codes can even 



asymptotically achieve capacity in the BSC with an iterative decoding bit-flipping scheme iZOl , 
IBZ02[|BZ04II . In these works, a worst-case analysis (for an adversarial channel) was performed 
as well. 

The best result for iterative decoding of such expander codes, reported by Skachek and 
Roth IISR03L implies a lower bound of p* = 0.0016 on the threshold of a certain iterative 
decoder for rate 0.375 codes. Feldman and Stein I1FS05II proved that LP-decoding can asymp- 
totically achieve capacity with a special family of expander Tanner codes. They also presented 
a worst-case analysis, which in the case of a code rate of 0.375, proves that LP decoding can 
recover any pattern of at most O.OOOSA^ bit flips. This implies a lower bound of p* = 0.0008 on 
the noise threshold. These analyses yield overly pessimistic predictions for the average-case 
(i.e., the BSC). Theorem [271 2) deals with average case analysis and implies that LP-decoding 
can correct up to 0.044A^ bit flips with high probability. Furthermore, previous iterative decod- 
ing algorithms for expander Tanner codes deal only with bit-flipping channels. Our analysis for 
LP-decoding applies to any MBIOS channel, in particular, it can be applied to the BLAWGN 
channel. 

However, the lower bounds on the noise threshold proved for Tanner codes do not improve 
the best previous bounds for regular LDPC codes. An open question is whether using deviations 
denser than skinny trees for Tanner codes can beat the best previous bounds for regular LDPC 
codes IIADS091 IHEllL In particular, for a concrete family of Tanner codes with rate |, it 
would be interesting to prove lower bounds on the threshold of LP-decoding that are larger 
thanp* = 0.05 in the case of BSC, and a* = 0.0735 in the case of BLAWGN channel. 



A Optimal Valid Subconfigurations in the Execution of N WMS2 

The description of algorithm NVv^MS2 as a dynamic programming deals with computation of 
optimal valid configurations and subconfigurations. In this appendix we define optimal valid 
subconfigurations and prove invariants for the messages of Algorithm NVv^MS2. 
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(a) Substructure T^-ltv for ^ / ^ /i - 1. (b) Substructure T^!^c for ^ ^ ^ /i - 1. 
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Figure 2: Substructures of a path-prefix tree T^^{G) in a dynamic programming that computes 
optimal configurations in T^^{G). 

Denote by T^'^J a path prefix tree of G rooted at node v with height 21 + 2 such that all paths 



must start with edge {v, G) (see Figure [2(a)] ). Denote by T^^'^ a path prefix tree of G rooted at 



node G with height 21 + 1 such that all paths mast start with edge (C, v) (see Figure [2(b)| ). 

Consider the message fJ-c^v ■ ^^ ^^ determined by the messages sent along the edges of 
j-2i+2(^Q^ that hang from the edge {v,G). We introduce the following notation of this subtree 
(see Figure [3]). Consider a path-prefix tree %^^{G) and a variable path p such that (1) p is a 
path from root r to a variable node v, (2) the last edge in p is (C, v) for G' ^ C, and (3) the 
length of p is 2{h — / — 1). In such a case, TqXI^ ^^ isomorphic to the subtree of Tj^ hanging 
from p along the edge {p^p o (t>, C)). Hence, we say that TqXI is a substructure of T^^{G). 
Similarly, if there exists a backtrackless path g in G from r to C with length 2{h — I) — 1 that 
does not end with edge (f , C), we say that T^^c ^^ ^ substructure of %^^{G). 

Let T^ub denote a substructure 7^^^ or TjflXc- ^ binary assignment z e {0, l}l"'^('^="b)l to 
variable paths V(7^ub) is a valid subconfiguration if it satisfies every parity-check path q E T^ub 
such that |g| ^ 1. We denote the set of valid subconfigurations of T^ub by config(7Iub)- 

Define the weight of a variable path q E V(7^ub) with respect to level weights w = 
{wi,...,Wh) ER^hy 



yVsuh(J'suh,q) 



n 



1 



UJh-l-l+l\q\/2-] 

degcitiq)) ■ ,^^ ,f,Vwr deg^ (t(g')) - 1 ' 

i}'eProfix+{(j)nV(7;ub) 



The weight of a valid subconfiguration z for a substructure T^ub is defined by 

yVsuh(Tsuh,z) = ^ At(g)(zg) ■ Wsub(7;ub,g)- 

{'2ev(r,ub)ll'2l^i} 
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%^^{G) ,M 




2h 



Figure 3: 7^!|^ as a substructure isomorphic to a subtree of the path-prefix tree T^^' 



-21+2 



Define minimum weight of substructures T^^c ^"^ '^c-¥v fa^ '^ ^ {0, 1} as follows. 



>VsT(V-7j,«) = min{W.,b(r,2^+J,^)|zGconfig(r,^_^+J),;^(c,.) = a}, and 



minf'-j-2l+2 



WsTCT^: 



min{Wsub(rc'!t',^) I z e config(7^!t').^(-) = «}• 



The minimum weight substructures satisfy the following recurrences. 
Proposition 39. Let a e {0,1}, then 
1. for every 1 ^ I ^ h — 1, 

Wh-l 

T ■ K{,a] + ■ 

C'&Jsf{v)\{C} 



ycsijis.a) = ^ . ua) + 5^^^ . E vv:r(r^,'-'«.a) 



2. /or every ^ I ^ h — 1, 



«eAr(C)\{i'} 






X e {0, l}'^^SG(^\||s||ieven,x^ 



The following claim states an invariant over the messages fic^^{a) and ^''^_^(j{a) that holds 
during the execution of NWMS2. 
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Claim 40. Consider an execution o/NWMS2(A(0), A(l), h, w). Then, for every ^ I ^ h — 1, 

/^?ic(«) = ^s:^ir:%',^)^ and 

Proof. The proof is by induction on I. The induction basis, for / = 0, holds because fil_^ci^) ~ 
yV^{biX!^c^ a) = ^^"''', . At,(a) for every edge {v, C) of G. The induction step follows directly 
from the induction hypothesis and Proposition [391 D 



B Proof of Lemma 32 



Proof We prove the lemma by induction on the difference I — s. We first derive an equality for 
Ee~*^' and a bound for Ee~*^' . Since Yi is the sum of mutually independent variables, 

Ee"*^' = (Ee-*'^'^)(Ee-*^'-l)'^^ (36) 

By definition of Xi we have the following bound, 

g-tX, ^ g-tE,li minimi {y/'>:is;is;dij} 

d' 

SC[d'j^]:\S\=d' i€S 

By linearity of expectation and since {Y^ }^^^ are mutually independent variables, we have 

Ee-*^'^ ("^AfEe-'A". (37) 

By substituting (l36l) in (l37l) . we get 

Ee-*^' ^ f Ee-*^'-) ' f^^'A (ec-'^A , (38) 



which proves the induction basis where s = / — 1. Suppose, therefore, that the lemma holds for 
/ — s = i, we now prove it for / — (s — 1) =2 + 1. Then by substituting (l38l) in the induction 
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hypothesis, we have 



Ee"*^' ^ ( Ee~*^' 



{d'^-d')^-^ l-s-l 






< 



Ee" 



-tXs 



fc=0 



^ I f Ee-*"'^^ 



d/,(4-rf')'-" «~s-l 



A;=0 



d', 



n ; (Ee-*--) 



d' 



Ee" 



-tx. 



id'^-d')l-^+^ l-s ^ ^^, ^ ^_^ (d'^.d')'' 



k=0 



n ; (E---")" 



(^.d')^' 



which concludes the correctness of the induction step for a difference of / — s + 1. D 



C Proof of Lemma 35 



Proof. By Lemma [32l we have 



]Eg-t^h-i < ( Ee"*^= 



{d'L-d') 



h-s-l 



d' 



; (Ee--)' 



(d'^-d')''-"-!-! 



Note that Ee ^''^ is minimized for e*'' = \/p{l — p). Hence, 



^(,-t^h^i < ( Ee 



-tx. 



{d'^-d')^-''-^ 



d' 



,f (2VRr^)'^' 



(4.d')fe-°-i~ 
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Ee" 



tXs \ "ii 



M(2v^^(rr^)' 



d' \ "l 



1 (4-d')''"*"l 



d' -d'-l 



d 



d' 



M(2vRr^) 



d' 



Let a ^ mini^o JEe'^^^ ({^'fj {2^p{l-p)) 



^,\ d^TdTTT 



. Let t* = arg min^j^Q Ee ^^^ , then 



Using Lemma [3TI we conclude that 
and the lemma follows. 



d' 
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